Efficient Intra-Rack Resource Disaggregation for HPC Using Co-Packaged DWDM Photonics

The diversity of workload requirements and increasing hardware heterogeneity in emerging high performance computing (HPC) systems motivate resource disaggregation. Resource disaggregation allows compute and memory resources to be allocated individually as required to each workload. However, it is unclear how to efficiently realize this capability and cost-effectively meet the stringent bandwidth and latency requirements of HPC applications. To that end, we describe how modern photonics can be co-designed with modern HPC racks to implement flexible intra-rack resource disaggregation and fully meet the bit error rate (BER) and high escape bandwidth of all chip types in modern HPC racks. Our photonic-based disaggregated rack provides an average application speedup of 11% (46% maximum) for 25 CPU and 61% for 24 GPU benchmarks compared to a similar system that instead uses modern electronic switches for disaggregation. Using observed resource usage from a production system, we estimate that an iso-performance intra-rack disaggregated HPC system using photonics would require 4x fewer memory modules and 2x fewer NICs than a non-disaggregated baseline.Comment: 15 pages, 12 figures, 4 tables. Published in IEEE Cluster 202

Silicon photonic 2.5D integrated multi-chip module receiver

We demonstrate the first 2.5D integrated, wavelength division multiplexing, silicon photonic receiver. The multi-chip module utilizes a silicon interposer to integrate the four-channel photonic cascaded microdisk receiver with four electronic transimpedance amplifiers

Ultralow-crosstalk, strictly non-blocking microring-based optical switch

We report on the first monolithically integrated microring-based optical switch in the switch-and-select architecture. The switch fabric delivers strictly non-blocking connectivity while completely canceling the first-order crosstalk. The 4×4 switching circuit consists of eight silicon microring-based spatial (de-)multiplexers interconnected by a Si/SiN dual-layer crossing-free central shuffle. Analysis of the on-state and off-state power transfer functions reveals the extinction ratios of individual ring resonators exceeding 25 dB, leading to switch crosstalk suppression of up to over 50 dB in the switch-and-select topology. Optical paths are assessed, showing losses as low as 0.1 dB per off-resonance ring and 0.5 dB per on-resonance ring. Photonic switching is actuated with integrated micro-heaters to give an ∼24  GHz passband. The fully packaged device is flip-chip bonded onto a printed circuit board breakout board with a UV-curved fiber array

The role of integrated photonics in datacenter networks

Datacenter networks are not only larger but with new applications increasing the east-west traffic and the introduction of the spine leaf architecture there is an urgent need for high bandwidth, low cost, energy efficient interconnects. This paper will discuss the role integrated photonics can have in achieving datacenter requirements. We will review the state of the art and then focus on advances in optical switch fabrics and systems. The optical switch is of particular interest from the integration point of view. Current MEMS and LCOS commercial solutions are relatively large with relatively slow reconfiguration times limiting their use in packet based datacenter networks. This has driven the research and development of more highly integrated silicon photonic switch fabrics, including micro ring, Mach-Zehnder and MEMS device designs each with its own energy, bandwidth and scalability, challenges and trade-offs. Micro rings show promise for their small footprint, however they require an energy efficient means to maintain wavelength and thermal control. Latency requirements have been traditionally less stringent in datacenter networks compared to high performance computing applications, however with the increasing numbers of servers communicating within applications and the growing size of the warehouse datacenter, latency is becoming more critical. Although the transparent optical switch fabric itself has a minimal additional latency, we must also take account of any additional latency of the optically switched architecture. Proposed optically switched architectures will be reviewed.This item from the UA Faculty Publications collection is made available by the University of Arizona with support from the University of Arizona Libraries. If you have questions, please contact us at [email protected]

Structured Errors in Optical Gigabit Ethernet

Abstract. This paper presents a study of the errors observed when an optical Gigabit Ethernet link is subject to attenuation. We use a set of purpose-built tools which allows us to examine the errors observed on a per-octet basis. We find that some octets suffer from far higher probability of error than others, and that the distribution of errors varies depending on the type of packet transmitted.

Impact of photonic switch radix on realizing optical interconnection networks for exascale systems

Abstract: We investigate the realization of large scale, 100,000-node optical interconnection networks with photonic switch fabrics of varying radices. Although such interconnection networks are realizable with 16-radix switches, radices greater than 40 provide a significant advantage. 1